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Abstract 

In this paper we study the quantization stage that is implicit in any compressed sensing signal ac¬ 
quisition paradigm. We propose using Sigma-Delta (EA) quantization and a subsequent reconstruction 
scheme based on convex optimization. We prove that the reconstruction error due to quantization de¬ 
cays polynomially in the number of measurements. Our results apply to arbitrary signals, including 
compressible ones, and account for measurement noise. Additionally, they hold for sub-Gaussian (in¬ 
cluding Gaussian and Bernoulli) random compressed sensing measurements, as well as for both high 
bit-depth and coarse quantizers, and they extend to 1-bit quantization. In the noise-free case, when the 
signal is strictly sparse we prove that by optimizing the order of the quantization scheme one can obtain 
root-exponential decay in the reconstruction error due to quantization. 


1 Introduction 

Compressed sensing (CS) [TI] [50] is a signal acquisition and reconstruction paradigm that is based on 
exploiting a fundamental empirical observation: various types of signals admit (approximately) sparse rep¬ 
resentations with respect to certain bases or frames. For compressive acquisition of a signal x C one 
collects relatively few, say m ^ N, inner products of the signal with well-chosen measurement vectors such 
as appropriate random vectors, say b* € , i € {l,...,m}. These inner products yield m compressive 

measurements yi = (bi, x) + rji , with f C {1,..., m} and m ^ N. Here rji represents measurement noise. To 
reconstruct a good approximation to x from these compressive measurements, one uses some sophisticated 
non-linear decoder, typically based on convex optimization or on greedy methods. Perhaps the most popular 
reconstruction method uses ^i-norm minimization. Specifically, x is approximated with x, the solution to 

min ||z||i subject to ll^z — j/lb < e. ( 1 ) 

Here $ is an m x V matrix with (pi as its ith row, y = (y^)™ i is the vector of acquired measurements, and 
e is an upper bound on the .^ 2 -norm of the noise-vector 77 = {rn)YLi- 

Vectors that are fc-sparse have at most k non-zero entries, i.e., they belong to the set := {x G 
|supp(x)| < k}. Compressed sensing results are typically formulated in terms ak{x)i^ := min ||x — u||i, 

called the best k-term approximation error of x in ii, which is a function that measures how close x is to 
being /c-sparse. It has been shown, e.g., mills], that for a wide class of random matrices (including those 
with i.i.d. standard Gaussian or ±1 Bernoulli entries), with high probability on the draw of the matrix and 


uniformly on x, the solution x to ([T]) satisfieE0 


||x 


xh < Cl 
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when m > C 2 k log {N/k). Accordingly, any fc-sparse x S can be exactly recovered from m N 
measurements. Furthermore, the solution x of © provides an approximation of x that is within the level 
of measurement noise (i.e., robust to noise) and almost as good as the best fc-term approximation of x in 
(i.e., stable with respect to model mismatch). The above is a concise description of some of the foundational 
results in compressed sensing. Note that the compressed sensing literature has grown extensively in recent 
years. For example, it includes work on expanding the classes of random matrices from which <i> can be 
drawn (e.g., [MIIIZIIIIII]), and on devising highly efficient reconstruction algorithms A so that A($x) = x 
whenever x G and so that ||A(d>x + e) — x ||2 is small otherwise (e.g., [42l [161 [6l [2T1149] ). It also includes 
work on incorporating additional structural assumptions or prior knowledge on the underlying signals to 
improve reconstruction accuracy or decrease the number of measurements (e.g., laiMiiii]). 


1.1 The quantization problem in compressed sensing 

Compressed sensing was named so mainly because of its potential to combine sensing (acquisition) with 
compression (source coding) - see, e.g., the discussion in Section I of [20]. However, most of the literature, 
as we briefly reviewed above, ignored analog-to-digital (A/D) conversion or quantization. Accordingly, 
compressed sensing emerged mostly as a dimension reduction technique rather than a scheme for compression. 

Here, we focus on a compressive data acquisition model consisting of a measurement stage that we have 
outlined above, followed by a quantization stage where the measurements are replaced with finite bit streams, 
and a reconstruction stage where the signal of interest is approximated from the quantized measurements. 
The set of signals of interest contains all bounded k-sparse signals in where k N and is denoted by 
:= {x : X G E^, ||x ||2 < /r}. We are also interested in bounded compressible signals, i.e., signals that 
can be well approximated in E^’^. We will denote the union of these sets by X. Let x G A and let y = $x 
be the compressive samples of x, where $ denotes an m x N compressive measurement matrix. We define 
a quantization map Q via its action {Qy)i = Qi, where qi G A, and A is a finite set called the quantization 
alphabet. Thus, the elements of A can be assigned finite-length binary labels, amenable to digital storage, 
and one example is the “1-bit” alphabet A = { — 1,1}. Given the quantized compressive measurements 
Q(<l>x), we want to recover a good approximation x of x via some reconstruction map Aq : A™ i—> with 

X = Aq(Q(<1>x)) that ensures that the reconstruction error ||x — x|| is small. 

1.1.1 Entropy numbers and optimal reconstruction error 

To measure the performance of a practical scheme consisting of the three stages discussed above (i.e., mea¬ 
surement, quantization, and reconstruction) we will examine the tradeoff between rate (total number of bits) 
and distortion (reconstruction error). Ideally, one seeks schemes that approach the optimal error decay for 
the class of signals at hand, say X = E^’^. In fact, the optimal error is induced by the covering number 
Af{X, II • II, e), defined as the minimal number of balls, centered in X, and of radius e (measured in the norm 
II • II) whose union contains X. If we were to assign each of the centers a binary label, this would be an 
optimal encoding of X, yielding an approximation error of e while using 7^*(e) = log 2 {M{X, || • ||,e)) bits 
(see, e.g., [40] 1. Inverting this function we obtain e*{TZ), the optimal error as a function of the number of 
bits. For example, for X = E^’^ a volume argument reveals the bound e*{TZ) > Moreover, this 

bound is essentially achievable for large enough TZ, by simply using log 2 {^) bits to encode the location of 
the non-zero coefficients of x G A and using the remaining bits to directly encode the non-zero entries. We 
note that such an approach is not available to us, as we only have access to the vector of measurements $x - 

* Since we consider CS matrices with unit-variance entries as opposed to those with unit expected-norm columns, the right- 
hand side of © contains a factor of affecting the noise term. 
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and not to x itself. Moreover, $a; must be quantized upon acquisition using analog hardware. Such hardware 
cannot invert the measurement system to recover x (otherwise there would be no need for quantization). 
Instead, one seeks quantization schemes that act on 3>a;, and reconstruction algorithms to obtain an estimate 
X from Q(<I>x). Below, we summarize some of the quantization and reconstruction strategies that have been 
proposed in the context of compressed sensing. 

1.1.2 Memoryless scalar quantization -with standard reconstruction techniques 

Perhaps the most intuitive approach to quantization is the so-called memoryless sealar quantization (MSQ). 
Here one replaces each measurement yi with the element of the quantization alphabet that best approximates 
it. Formally, a scalar quantizater Q _4 : M —> .4 is defined via 

Qa{z) - z\. (3) 

v^A 

Suppose that we take m measurements of signals in df, MSQ maps the vector of measurements to by 
acting independently on each entry: 

: ^{X) ^ (4) 

y !-)■ g, where = (5^(yi), i=l,...,m. 

While attractive for its simplicity and widely assumed in the mathematical treatment of signal acquisition 
methods, MSQ is generally not optimal when one oversamples, i.e., one has more measurements than needed 
for $ to be injective [4^ . This sub-optimality is because MSQ ignores correlations between measurements (as 
it acts component-wise). Still, at first glance this sub-optimality is well-tolerated in the compressive sampling 
framework because this framework does not rely on oversampling. In fact, MSQ yields near-optimal rate- 
distortion guarantees when one is permitted to take a minimal number of CS measurements and to reduce 
the reconstruction error (i.e., the distortion) by increasing the size of the quantization alphabet [12) . 

On the other hand, such an approach may not be desirable, or even practical, in applications. It does not 
permit improving the accuracy of the reconstruction once the acquisition hardware, thus the quantization 
alphabet, is fixed. This is observed in the approximation error bound ([2]), which in a quantization setting 
does not guarantee that the error decays as more measurements are taken. For example, consider the scalar 
quantization setting where every measurement t/i is replaced by the closest element from 

Al := {±(2 j-l)d/2,je {!,...,L}}, 

the so-called 2L-level midrise alphabet with step size S. Here, when yi is appropriately bounded, we have 
\yi — qi\ < 5/2 and so one must take e = sjrnbl2 in ([T]) and in the error estimate ([2]). This shows that using 
simple MSQ coupled with standard reconstruction algorithms only guarantees a reconstruction error that 
scales with d, the step-size in the quantization alphabet. It does not guarantee any error decay with to. 
Consequently, in this set-up our only resource for obtaining small reconstruction error (due to quantization) 
is to use a small quantization step-size b, thus a large alphabet. Importantly, this typically implies a high 
implementation complexity in hardware. 

1.1.3 Modifications to MSQ, and alternative reconstruction techniques 

To overcome this shortcoming of MSQ, one may devise more sophisticated reconstruction algorithms to 
improve the error behavior as the number of measurement increases. For example, this idea was explored in 
[301 |3T] where the constraint of o is replaced by an constraint with p > 2. By choosing p optimally, 
[30] shows that the reconstruction error decays as (logm)”^/^ as to increases. A different reconstruction 
algorithm based on message passing was proposed in |33j with an error that is empirically observed to decay 
like 1 /to. In fact, lower bounds obtained in the frame quantization setting |25j show that such an error 
rate is optimal for MSQ. The error associated with this quantization technique can at best decay linearly in 
the number of measurements, regardless of the reconstruction algorithm. This is far from optimal, as the 
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optimal encoding (over all quantization methods) of sparse signals yields an error that decays exponentially 
in m (as we discussed above). Note that one can essentially match the optimal bound by introducing dither 
and modifying the scalar quantizer so that non-contiguous intervals map to the same element of ^ [S]. 
However, this does not overcome the increased hardware complexity associated with a large alphabet, as the 
hardware must still distinguish between small intervals in order to map them to the appropriate element of A. 
Additionally, this approach has theoretical guarantees only for a combinatorially complex, i.e., impractical, 
decoder. 

Alternatively, one may consider improving the rate-distortion tradeoff associated with MSQ by efficiently 
encoding its resulting bit-stream. This entails finding a simple map from A^ to a smaller codebook C, with 
log \C\ <C mlog |A|, and a reconstruction algorithm that guarantees sufficient error decay with m. However, 
we do not know of any such encoding/reconstruction schemes with theoretical guarantees on the decay of 
the worst-case error. 

1.1.4 One-bit MSQ and its variations 

Despite the shortcomings of MSQ, its extreme case of one-bit quantization has been the focus of several 
contributions. Here, one simply replaces every measurement yt by sign(j/i) and as a consequence loses all 
information about the magnitude of y, hence x. Nevertheless, the focus in such a set-up is approximating x 
up to the lost magnitude information. The first results in this direction |101132j derived lower bounds on the 
achievable reconstruction accuracy and proposed some heuristic algorithms for estimating x. Afterwards, 
[45] proposed estimating as the minimizer of a particular convex optimization problem and showed 

error decay rates of ■ To remedy the loss of magnitude information, [33] recently proposed quantizing 

yi to sign (jji -|- Ti) for a fixed vector of thresholds r and presented associated reconstruction methods. The 
corresponding reconstruction error decay rate associated with approximating x, namely was also 

derived in [34] . 

In another recent work, [2] proposed quantization schemes that, like the EA schemes we focus on, 
adaptively update a quantization threshold as the measurements are taken. That is, in the one-bit case they 
propose quantizing yi to sign {yi -|- Ti) albeit now the thresholds are adaptively chosen. These schemes yield 
exponential error decay in m, but unlike EA schemes, require very sophisticated computation to assign q. For 
each update of the thresholds, an £i-norm optimization problem is solved (or an iterative hard-thresholding 
algorithm is run). Furthermore, the scheme entails communicating analog quantities (the current thresholds) 
to a digital computer (to solve the optimization problem). It also entails halting the measurement process 
until the optimization ends. 

1.1.5 Noise-shaping quantization methods for compressed sensing 

Recently, noise-shaping quantizers m have been successfully employed in the compressive sensing frame¬ 
work. Traditionally such quantizers, for example EA quantizers, are used for analog-to-digital conversion 
of bandlimited signals and they are designed to push (most of) the quantization error outside of the signal 
spectrum (see, e.g., |33])- Over the last decade it has been established that noise shaping quantizers, includ¬ 
ing EA quantizers as well as a new family of quantizers based on beta encoders m cf. [13] (not our focus in 
this paper) can be used in the more general setting of linear sampling systems associated with both frames 
and compressed sensing. 

We remark that EA quantization, like MSQ, is universal, i.e., it does not require or use information 
about the vectors (pi or x. Furthermore, it is progressive, so it continues to operate in the same way as 
new measurements arrive. In the case of frames, EA schemes ensure that the quantization error is close to 
the kernel of an appropriate reconstruction operator based on noise-shaping alternative dual frames called 
Sobolev duals [5JI31], see also [171I371I5S]. EA quantization schemes can also be used in the case of compressed 
sensing. Work preceding this paper mainly uses a two-stage reconstruction scheme [27], see also [321122], 
where in the first stage one estimates the support of the original sparse signal and in the second stage, one 
uses an appropriate Sobolev dual to reconstruct. 
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Above, we have provided an overview of various quantization and reconstruction approaches associated 
with compressed sensing. These approaches differ in their implementation complexity, in their reconstruction 
algorithms, and in the quality of their reconstruction guarantees. We refer the reader to [5] for a more 
comprehensive review of the literature on quantization in the compressed sensing context. In what follows, 
we focus on using EA schemes of order r to quantize compressive samples of sparse or compressible signals. 
The SA schemes we study are low-complexity and do not place much computational burden on the quantizer. 
We dedicate Section [2.31 to reviewing the prior work on EA-quantization in the compressed sensing context. 

1.2 Contributions 

We show that EA schemes allow accurate reconstruction from quantized CS measurements via convex 
optimization. Our work generalizes in several ways the results of [171137], which applied to sparse signals 
that satisfy a size condition on their smallest non-zero entries. In particular, the convex optimization problem 
we propose allows us to remove the size condition and to obtain results that apply for general signals in noisy 
scenarios. It also allows us to use small, even I-bit, quantization alphabets. If q denotes the EA quantization 
of the noisy compressed sensing measurements <I>x -I- rj where rj satisfies ||?7||oo < e, then we recover x from q 
by solving 


(x, r>) := argmin Ijzlli subject to ||ZI {^z + v — q )\\2 < ^{r)\/rn 

and ||i ^||2 < e^/m. (5) 

Here, D denotes the m x m difference matrix with entries Di^i = 1, Hi+iy = — I and Dij = 0 when 
j ^ {*,* — !}■ Additionally, r denotes the order of the EA scheme, and 7 (r) is a constant associated with 
the EA scheme, see Section (2731 Our main result follows. 

Theorem 1. Let k,(.^m,N he integers satisfying m > £ > ck\og{N/k) and let <I> he anmx N sub-Gaussian 
measurement matrix. With high probability on the draw o/$, for all x G , the solution x of ^ where q 
is the EA quantization of ^x + lu (with ||a;||oo < satisfies 

P-* ^ C ((!=)—. + + ,8, 

where c, C are constants that do not depend on 

We note that Theorem [3] is a more precise version of Theorem [T] with all the probabilities and constants 
made explicit. Our main result shows that EA quantization of compressed sensing measurements has the 
following desirable features. 

• Polynomial error decay: By ([S]), the recovery error associated with quantization decays polynomially 
in the number of measurements. Polynomial error decay outperforms the lower bound on the error 
decay associated with memory-less scalar quantization, which is only linear. 

• Generality of measurement systems: The result holds for a broad class of sub-Gaussian com¬ 
pressed sensing matrices. 

• Stability and robustness: It holds for arbitrary signals in the presence of measurement noise. 

• Coarse (even 1-bit) quantization: It applies even in the case of 1-bit quantization, when appro¬ 
priate EA schemes, such as those of | 26 l E] , are used. 

In addition to the above features. Theorem [1] which holds for various choices of ^ G [cklog{N/k),m], 
has several important implications. Some of these implications, listed below, can be fleshed out by a careful 
optimization of appropriate parameters in ([3|) under various signal and noise models. 
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• Root-exponential error decay for fc-sparse vectors (Corollary ITT]) : By optimizing the order of 
the SA quantization scheme, one can show that the reconstruction error decays root-exponentially in 
the number of measurements in the absence of measurement noise. That is, defining A := ^ log’^/fc) j 

This matches the best known bounds (in the absence of further encoding for the finite frames case 
of SA quantization |5S] . 

• Error decay for compressible vectors (Corollary [T^ : We model a compressible vector x as an 

element of the unit ball of a weak £p space w£p with p € (0,1). That is, the coefficients of x satisfy 
|2;(j) I < where is the jth largest-in-magnitude entry of x. The reconstruction error associated 

with such vectors, in the noise-free scenario, satisfies 

(l/p-l/2)(r-l/2) 

flogN\ 

lk-a;||2 < ( ) 

We note that the lower bound on the reconstruction error, e, associated with quantizing vectors in ip 
using R bits, with R £ [log 2 N, A], is (see, e.g., [5D] i 

^ log(A/R + l) y/P-^/^ 

Thus, for a fixed p and using a coarse SA quantizer (for example, as in [26)1. increasing r allows us to 
approach the optimal error rate for compressible signals. 


2 A/D conversion for compressed sensing by EA quantization 

2.1 Basics of SA quantization 

SA quantizers were introduced by Inose and Yosuda [28] in the context of analog-to-digital conversion for 
oversampled bandlimited functions. Because SA quantization schemes are robust with respect to certain 
circuit implementation imperfections, they have seen widespread practical use since their introduction, see, 
e.g., [43] . Moreover, the mathematical literature on SA quantization has significantly matured over the last 
15 years, see, e.g., [miMi na ED- For example, as we will discuss below, SA schemes were shown to be 
more effective than MSQ in utilizing redundancy when quantizing redundant frame expansions, see, e.g., 

HIZIEIIMIEZ], cf. gEj. 

Given a (hnite or infinite) sequence y := {yi)iex, the standard first-order SA quantizer runs the following 
iteration for i G I: 


Qi = QAiVi + Ui-i) (7) 

Ui = Ui-i +y^- qi. 

Here is the scalar quantizer defined in Q. More generally, an r*^-order SA quantization scheme with 
quantization rule p : —?► K and scalar quantizer Qj, computes q £ via the recursion 

qi = Qa {P{yi, Uj-l, Mj-2, • • ■ , Ui-r)) , (8) 

Ui = yi-qi-^y\{-iyui-j (9) 

tit was shown in m that by incorporating an extra encoding stage, exponential accuracy in the bit-rate can be achieved. 
Similar ideas, with similar results, can be applied in the compressed sensing scenario, but we leave this topic for a different 
paper. 
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for i € I. For example, in the compressed sensing and finite frame scenarios (see below) yi = 
i = 1, In other words y = and the relationship between x, u, and q can be concisely written using 

matrix-vector notation as 

D'^u = <^x-q. (10) 

Here the matrix D is the m x m first-order difference matrix with entries given by 

( I Hi = j 

-1 if^ = j + l . (11) 

[ 0 otherwise 

Due to the role that u plays in (HOI) and in controlling the reconstruction error, we will focus on stable rth 
order schemes. These are schemes for which (jS]) and ([9|) result in 

\\u\\oo < 7 := 7(r) 

for all m G N, and for all y G K™ with ||?/||oo < 1- Clearly, one can scale the definition of stability to replace 
the upper bound on ||y||oo by an arbitrary constant y. However, we require that 7 : N 1 —>■ R+ be independent 
of both m and y. Stable rth order SA schemes with 7 (r) = Oir'^) exist [551 H] even when ^ is a 1-bit 
alphabet. We will focus on two important families of stable SA schemes of arbitrary order. 

The rth-order greedy SA quantizer 

The rth-order greedy EA quantizer is obtained by setting 

pid^n— 1 ; • ■.; '^n—r^ Ui) • — ^ ^ ( 1 )"^ ( ' ) 

j=i V.?/ 

in ([5]). This choice results in a stable EA quantizer if we use the 2iF-level midrise alphabet with 
K > -I- 2’’ -f 1. The size of the alphabet associated with the greedy rth-order EA scheme grows 

exponentially in r. On the other hand, the stability constant 

7 (r) = 5/2 


remains fixed as r grows. 

The rth-order coarse EA quantizers 

A different tradeoff between the size of the alphabet and the stability constant arises with the so-called 
rth-order coarse EA quantizers. These are rth-order EA quantizers with a fixed alphabet A (for example 
A = {±1}) and were hrst constructed in [T^. Adopting the scheme of [19], we have 

qn = A Vn) (12) 

v^ = {h* v)n +yn- qn, 

where * denotes convolution. In [26|[T9], it was shown that by choosing h carefully one obtains a stable EA 
scheme when || 2 /||oo < y as above. In fact, when ||/i||i < 2K — 2y/5 one can rewrite p2p in the form of ([ 8 |) 
and ([9|) via a change of variable. One can then show that the corresponding scheme is stable with constant 

7(r) = 

for some constant C 3 that only depends on the alphabet A^ and y. 
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2.2 EA quantization for frame expansions 

As we mentioned above, while initially proposed for quantizing bandlimited functions, recent work on SA 
quantization has shown it to be well suited for quantizing finite frame expansions (e.g., UlilEllSSlil. In the 
SA quantization context, consider full-rank matrices E € m > k selected from certain classes, such 

as harmonic frames [1], piecewise-smooth frames [S], Sobolev self-dual frames [35], Gaussian frames m, 
sub-Gaussian frames m- Let X G and suppose that q is obtained by quantizing Ex using an rth-order 
SA quantizer. A typical results states that E admits a specific left inverse (equivalently, a dual frame), called 
its rth-order Sobolev dual and denoted by Fsoh,r{E) such that the reconstruction error ||Fsob.r(£’)9 — x \\2 
decays like Here A := m/k is the oversampling rate. In fact, this error decay is typical of the 

deterministic frames mentioned above. On the other hand, the error for random frames is 
with high probability that depends on a G (0,1). 

This error decay is remarkably similar to the error decay in the bandlimited case, which is also inverse 
polynomial in the oversampling rate. In the bandlimited case, this allows improving the reconstruction 
accuracy by sampling at a faster rate. In the finite frame setting one can also improve the reconstruction 
accuracy without changing the quantization hardware by simply increasing m, i.e., collecting more samples. 
This allows decreasing the reconstruction error beyond the accuracy of the quantizer itself. 

2.3 EA quantization for compressed sensing: an overview of existing results 

As stated in the Introduction, inn [nil [33 showed that by using EA schemes instead of MSQ, finite frame 
quantization techniques (e.g., [HOEKMIIST]) can be leveraged to the compressed sensing case. A key insight 
is that knowing the support of an underlying sparse vector reduces a compressed sensing quantization problem 
to a frame quantization problem. To be precise, let x G with supp(x) = T and let G be a 

compressed sensing measurement matrix. Then, we have 

y = ^x y = ^tXt- 

Accordingly, |27j proposed using an rth order EA-scheme, and a sufficiently fine, fixed alphabet, to quantize 
compressive samples of strictly sparse signals whose smallest non-zero entry is bounded away from zero. 
Moreover, |27j introduced an associated two-stage algorithm for recovering the signals from the quantized 
compressed sensing measurements. In the first stage of the algorithm, the signal support is estimated. Here 
one uses a standard robust compressed sensing decoder, e.g., o, to obtain a coarse estimate of x, say x. 
The support estimate T is then the subset of {1, ...,iV} on which the s largest (in magnitude) entries of x 
reside. When $ is a Gaussian random matrix, m showed that T = T with high probability if the smallest 
non-zero entry of x exceeds a constant multiple of the quantizer step size S. Having revealed the support T, 
linear reconstruction using frame theoretic methods recovers the signal via 

X = Tgob,r(d*^)(7. 

The main result of |27] was that in this setup (where the signals are strictly sparse and bounded away from 
zero, and the alphabet is sufficiently large) the reconstruction error associated with an rth order EA-scheme 
decays like with high probability depending on the parameter a G (0,1). These results were 

generalized to the case of compressed sensing matrices with sub-Gaussian entries in m and sub-Gaussian 
columns in [22]. 

finite frame [131135) for is a collection of m (m > k) vectors that spans R^. As such it can be identified (up to 
permutations) with an m x k matrix, say E, whose rows are the frame vectors. 
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EA quantization for compressed sensing: a novel one-stage re¬ 
construction method 

3.1 The algorithm 

An important shortcoming of the two-stage algorithm describe in the previous section is its need for an 
accurate estimate for the support of x. This restricts the use of the algorithm to sparse vectors with 
smallest (in magnitude) entry bounded away from zero. Moreover, it restricts the quantizer step size 5 to 
be (up to a constant) as small as the lower bound on the non-zero entries. In turn, this precludes the use 
of coarse quantization with small alphabets. An additional shortcoming of the two-stage algorithm (with 
reconstruction using Sobolev duals) is its lack of robustness to additive noise. 

Instead of the two-stage algorithm, we propose estimating x as the solution to a convex optimization 
problem that takes q = where |l?7||oo < e, as input. It produces an output x via 

(i, i>) := arg min || 2 ;||i subject to \\D~'^{^z + v — ( 7)||2 < 

and \\v \\2 < €y/m. (13) 

Here 7 (r) is the stability constant associated with the SA scheme. As discussed in Section ETTl when using 
a midrise quantization alphabet with a greedy quantization scheme, 'y(r) = 1/2. Furthermore, when using 
the coarse scheme in (IT^ . 7 (r) = C'Jr’’. One may also estimate x as a solution to a related (and perhaps 
more natural) optimization problem, 

(x, z>) := arg min ||z||i subject to \\D~^{^z + iz — q)\\ao < 7 (x) 

and \\i '\\2 < tsfm. (14) 

We remark that both optimization problems are designed to find the vector with the smallest l\ norm 
that agrees with the quantized measurements and the boundedness of the non-quantization noise. The 
first constraint in each of m and (fHl) is motivated by the simple observation that the stability of the 
EA quantization schemes used implies that ||I?“’'($x + r] — (?)||oo = Halloo < which in turn implies 

that ||u ||2 < ^(r)y/m. The remainder of the paper is dedicated to proving that the above reconstruction 
algorithms have none of the shortcomings of the two-stage algorithm. Our proofs will be for the optimization 
problem (fT^ but they apply equally to (fTdll . since any feasible point for (fT4l) is also feasible for (fT^ . 

3.2 Preliminaries 

We will need the following notation and results. 

3.2.1 Sub-Gaussian random matrices 

Below, we write x ~ 21 when the random variable x is drawn according to the distribution T) and A/'(0, cr^) 
denotes the Gaussian distribution with mean zero and variance a^. 

Definition 2. Suppose x ^ Hi and y ^ 1^2 are random variables that satisfy H(|x| > t) < KP{\y\ > t) for 
some constant K and for all t >0. Then x is said to be if-dominated by y. 

Definition 3. A random variable is sub-Gaussian with parameter c > 0 if it is e-dominated by A/'(0, c^) 

Definition 4. A matrix $ is said to be sub-Gaussian with parameter c, mean p,, and variance if the 
entries $ are indepent sub-Gaussian random variables with parameter c, mean /i, and variance 

Note that Sub-Gaussian random variables can be equivalently defined using their moments. Moreover, in 
the case of zero mean random variables, their moment generating functions can be used. See m for these 
definitions and proof of equivalence. 
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3.2.2 Some key results from the literature 

Next we review some useful results that are instrumental for the error estimates presented in the next section. 
We begin with a proposition from m which controls the restricted isometry constants of some matrices of 
interest. Here, given a matrix V we denote by the matrix formed by the first i rows of V. 

Proposition 5 ([37]). Let <i> be an mx N sub-Gaussian matrix with mean zero, unit variance, and parameter 
c, let V be an orthonormal matrix of size m x m. Fix some /3 > 0, some k £ Z"*" with k < 
some I £ IF' with t > CiklogN/k. Then, there is a pp such that 


p[ sup 5] <e 

yrC[Ar],|T|<fe f J 

where cp pp only depend on 6 and c. 

The next proposition provides useful bounds on the singular values of D~^. 

Proposition 6 ( [27jl. The singular values of the r-th order difference matrix satisfy 

(15) 

We will need the following two propositions, the first of which follows from Theorem 5 and Proposition 
8 in [23]. 

Proposition 7 ([23]). Let f,g £ <C^ , and <i> £ Suppose that <i> has S 2 k-RIP with 62 k < 1/9, then for 

any 1 < p < 2 , we have 

11/ - gh < C,kFp-F2\\<^>if - g)y + j^iWfh - II 5 II 1 + 2ak{g)i), 

where C 4 and C 5 are constants that only depend on 52k- 

Proposition 8. Let ^ be an m x TV, m > TV, sub-Gaussian matrix with mean zero, unit variance, and 
parameter c, then the largest and smallest singular values of $ obey 

P((Ti($) > y/m + a/TV + t) < e“^, and P (‘h) < y/rn — VN — t^ < e~^. 

where d > 0 is some constant only depending on c. 


3.3 Error estimates 

The following theorem provides an error bound on the estimate from (1131) . 

Theorem 9. Let $ be an mx N sub-Gaussian matrix with mean zero and unit variance, and let k and £ be 
in {1,, m}. Denote by ® stable rth-order scheme with alphabet and stability constant 7 (r). There 
exist positive constants c and p such that whenever m > £ > cfclogTV/fc the following holds with probability 
greater than 1 — exp(—cp/c log(TV/fc)) on the draw 0 /$ : 

Let X £ such that ||fl>x||oo < M < 1- Suppose q := where the additive noise rj satisfies 

IIpIIoo < e for some 0 < e < 1 — p. Then the solution x to CS]) satisfies 


p - x \\2 < Ce 


/mN -’’+ 1/2 

\j) 


5 + C7 


<Jk{x) 

y/k 



where Cq, C 7 , and Cg are explicit constants given in the proof. 


( 16 ) 
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Proof. Let {z,v) be an ordered pair that is feasible to (IT^ . and let 7 := "f{r)/5. Define u := D ^{^z + v — q), 
and p := (^^u, and note that 

||u ||2 < 'ySy/m, and ||p ||2 < S\/^. (17) 

By definition, u, p, q, z, and v have the relation 

— g = D^u + v = 
which upon defining H := [ 7 D’', |/] becomes 

- g = Hp. (18) 

Let UYjV'^ = H he the singular value decomposition of H, and denote by := V'^Tj~^U the pseudo-inverse 
of H. Now, apply to both sides of (fT 8 l) to obtain 

H\^z-q) = H^Hp = VV'^p. (19) 

Then, (flTll and IT^ together imply that for any z that satishes the constraint in (IT^ . we have 

||i 7 ^(d>z-g )||2 ( 20 ) 

In particular, both x and x satisfy the constraint in m, so by the triangle inequality 

\\H^^{x - x )\\2 < \\H^^x - g )||2 + \\H^^x - g )||2 < 26V^- ( 21 ) 

On the other hand, the singular values of can be expressed in terms of those of D’’. Thus, using 
Proposition [ 6 ] we have 


erg 


(i7t) = 


~2 2 
7 




- 1/2 


> 7 " 


3TTri\ 


) 2r 

+ (l) 


- 1/2 


( 22 ) 


Denoting by ^ and the first £ rows of E ^ and $, and by Ug the first £ columns of U, we may now use 
the above bound on ag{H"^) along with (HU, to obtain 

2(5v^ > ||i7^$(x - i)||2 = ||P^E-it/$(a; - x)\\2 > \\^-^U<i>{x - x))\\2 (23) 

> \\Ef^Ug^i{x - x )\\2 > ag{H^)\\Ug^g{x - ®)|| 2 . 

Setting $ := -^Ug^g, we thus have 

25\/2m> ag{H^)^f£\\^g{x — x)\\2. (24) 

Moreover, by Proposition [5] we know that for any /3 < 1/9 there exist constants c and p, (that depend on /3) 
so that with probability exceeding 1 — e~^^, $ has the restricted isometry property with constant /3, provided 
that £ > ck\ogN/k. So satisfies the hypotheses of Proposition [3 which we can now apply with p = 2 and 
with X and x in place of / and g. By first using the fact that ||5;||i < ||a;||i and then (j24p and (l22)l we have 

CTfc(a;)i 


Ik - x\\2 < C'4||d’(a; - x )\\2 + C 5 - 


Vk 


< 2\/2Ci5 


m 1 


<yk{x)i 
£ <Jg{W) ' y/k 


C. 


< 2V2Ci5jj ( 7 ' 




o'kix)i 

Vk 


< 




gfc (a^)i 

\fk 


< 2V2Ca^3^ttW^ ' 5 + 2v^C 47" Y^e + ^5^^^ 
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Setting Cq := 2^/^C'43’'7r’’r’'7(r), Cj := 2^/2Ci^{r), and Cg := Cg, the error bound (fTO is established. □ 

Remark 9.1. The bound dUl) shows that the reconstruction error due to the perturbation rj scales linearly 
with II 7 II 2 (e.g. by setting i = m). Such perturbations (commonly referred to as measurement noise) model 
a variety of phenomena, including circuit imperfections. On the other hand, EA quantization is robust 
to certain circuit imperfections (see, e.g., [HI, m) of which we now provide an example. Suppose that 
we use a 1-bit alphabet A = {—5/2,5/2.} and an associated scalar quantizer to perform 1st order EA 
quantization, and suppose further that Qa is “imperfect”. In particular, suppose that Qa{z) = sign( 2 ) 
whenever \z\ > p for some p € [0,1) but that Qa{^) can be ±1 otherwise, for example at random. In this 
case, the stability constant (see Section l^TTl) associated with the scheme simply becomes 5/2 + p (see [TTlfTB] !. 
As a consequence of our proof, 5 + 2p replaces 5 in (1161) and the contribution of p can be made arbitrarily 
small by taking more measurements. 

Below, we prove some important implications of Theorem 121 Corollary [TO] distinguishes different regimes 
for the reconstruction error, depending on the relationship between m, e, and 5. Corollary 1111 shows that 
choosing the optimal order of a coarse EA quantization scheme yields root-exponential error decay in the 
number of measurements. Corollary 1121 also dealing with the noise-free scenario, derives the error decay 
associated with EA quantization of compressible signals (as a function of m). 

Corollary 10. Let $ be an m x N sub-Gaussian matrix with mean zero and unit variance, and let k £ 
{l,...,m} with m > 2ck\ogN. Denote by ® stable rth-order scheme with alphabet A^j^. There exist 

positive constants c and p (as defined in Theorem w such that the following hold with probability exceeding 
1 - on the draw of 


Let X € such that ||$a;||oo < M < 1- Suppose q := + rf), where the additive noise 77 satisfies 

||''?||oo A e for some 0 < e < 1 — p. Define Ic '■= ^ minimizer of (ITOl) . We 

distinguish three regimes based on 1^ and m. 

(i) If ic < ck log N (the low-noise scenario), then 


\\x - x ||2 < Ce 


2ck log N/k 


-r+lD 




ck log N/k 


(25) 


(ii) If cfclogA < ic < rn (the intermediate-noise scenario) then 

(Jk{x) 


\x-x \\2 < Cg5-rA--r +Ct 


y/k 


(Hi) If ic iL ni (the high-noise scenario), then 


- x||2 < Ce5 + + Cge. 


(26) 


(27) 


Here Cq, C-j, Cg are the same as in Theorem\^ and Cg depends on r, Ce, and Cg. 
Proof. We have, from ([TO|) that 


\\x - x ||2 < Ce 


/m\ - 1 - 1/2 

\l) 


5 + C7 


\fk 



(28) 


provided that m > i > ck log N. 

The critical point of the right hand side in (1281) . viewed as a function of i, is a minimum, and given 
by := cfclogA < ic < m then we get (ITOl) by the following argument. Define 
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f{i) := fi{£) + / 2 (^) where fi(i) := Cq (y) i5 and f 2 {£) ■= While we may not directly 

substitute ic into f{£) as £ must be integer valued, we observe that there is an integer £* between £^2 and 
£c with 


f{£c) </{£*) = fi{£*) + f2{£*) 

</i(4) + /2(4/2). 

Otherwise if £c ^ [ck log N, m], the minimum of / is achieved at one of the two boundary points £\ = ck log N 
(corresponding to (l25ll i and £ 2 =m (corresponding to (|?7l) i. In particular, /(m) yields (|?7)l . while f{\£i]) < 
/i(2£*)+/(£t) yields (I2S1). □ 


Corollary 11 (Root-exponential accuracy). Let rth-order coarse EA quantizer with stability 

constant ^ir) = C^r ^6 as described in Section \2.1\ and suppose that the hypotheses of Theorem\^hold. Letx € 
be k-sparse such that ||$a;||oo < 1 and suppose q := ^ \ck log N/k] • 

The solution x to 0 satisfies 

\\x-xh<Ciie-^^^'^ 5, (29) 


with probability exceeding 1 — e cpfciogAf/fc^ Here Cio depends only on C 3 , defined after (US])) Cii, C 12 do 
not depend on m,k,N, and c,p are as in Theorem\^ 


Proof of Corollary \11\ In the case of the coarse rth-order EA quantizer with stability constant 7 (r) = 5 

and when the signal is sparse and the measurements are noiseless, (HU) reduces to 


/ui\ —'^+ 1/2 

\\£-x\\2<C4[j) 6 , 

= Ci3(37rC3)V’'(-) 5 (30) 

where C 13 = 2^/^C'4. Since this inequality holds for any £ > cklogN/k, it holds for £ = \ck\ogN/k~\ which 
yields 

P - x||2 < Ci3(3^C'3)V2’'A-"VA (5. (31) 

Next, note that for a fixed A (equivalently m) the right hand side of (l3T|) depends on the order of the EA 
scheme, r, and can be optimized yielding 

P — x\\2 < C'la min(37rC3)^r^’'A“''-\/A 5. (32) 

rGN 


As explicitly shown in [55], the critical point of the function f{t) = *-\/A is given by 

t* = e-i(37rC'3)-i/pi/2. 

Accordingly, setting r = [t*J and simplifying, we obtain 

p - a;||2 < <5 

Setting Cii = ^< 7136 ^( 371 ( 73 )^/^ and C 12 = (37r(73)“^/^ we arrive at (155)) . 


□ 


Next, we consider the case when the signal is compressible and the measurements are noise-free. In this 
special case, dm) reduces to 

P-*<A(f)""‘'7 + C,^^, (33) 

where £ and k can be chosen freely as long as m> £ > cklogN/k. Below, we optimize this upper bound for 
compressible signals, modeled as elements in weak £p space. Specifically, we say that x is in the unit ball of 
the weak £p space w£p if |a;(j)| < where is the jth largest-in-magnitude entry of x. 
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Corollary 12 (Compressible signals). Let $ be an m x N sub-Gaussian matrix with mean zero and unit 
variance, and let k and £ be in {1,, m}. Denote by Qsa ® stable rth-order scheme with alphabet Fix 
p £ (0,1) and set a := There exist positive constants c and p (as defined in Theorem\^ such that 

whenever m > Ci 4 clogN, the following holds with probability greater than 1 — exp(—cpm“ log^~“ N) on the 
draw of $ : 


For vectors x £ in the unit ball of w£p , let q := Q 2 a(^^)- Then the solution x to (I13|) with e = 0 


satisfies 


l/p-l/2 

||x - x\\2 < Ci5(5'-+i/p-i 


(l/p-l/2)(r-l/2) 

r+l/p-1 


^(c+l)log7V 

Here C 14 and Cis depend on r, t, 6 and are given explicitly below. 


(34) 


Proof. It will be notationally less cumbersome to work with t = 1/p. For any x in the unit weak iijt ball, 
its tail (measured in the £i norm) decays as 


N 


<Jk{x) < / 

fc+1 


Ms < 


t- 1' 


Inserting this bound and e = 0 into (ITCl) . we get 


p-a:||2 < Cg 


/mN “’■+ 1/2 

\j) 


.5 + 


C 7 
t - 1 




Note that for any hxed k, we can make the upper bound above smallest by choosing the smallest £ for which 
(HI holds, i.e., £ = \ck\ogN/k~\. With this choice, we now have 


||i - x \\2 < Ce 
<Ce 


\c k logiV/fc] 


-r+ 1/2 

6 + 


^7 p^-t+1/2 

t - 1 




-r+1/2 


c' fclogiV^ 


C 7 p-t+ll2 
t -1 


(35) 


Above, replacing c by c' := c + 1 allows us to remove the ceiling function, and we relaxed log N/k 
to logiV to make optimizing over k easier in the next step. To simplify the ensuing argument define 

fi{k) := Cie ( + kTogN ) ^ ^’^d / 2 (fc) := ^k-*+^A where Cw := niax C 4 (k) and Cit := 

\ * / kG[l,m/(clog N)] 

max C' 5 (fc). The critical point of f(k) := /i(fe) + f 2 {k), the right hand side in (1551) . is 

feG [l,m/(c log N)] 


kc 


/Cis / TO 

,5 ^c' logiV 


r-l/2N 


1 

r+t-1 


where Cis = Ci 6 (t-i)(r-i/ 2 ) • particular, this critical point is readily seen to be a minimum. However, 
for (1551) to hold, k must be an integer satisfying 1 < k < so we can not directly substitute kc in 

Nevertheless, we note that kc is a minimum, fi{k) is increasing with fc, and / 2 (fc) is decreasing with 
k. Moreover, if fee > 1 there exists an integer k* between kc and fcc/2 with 

f{kc)<f{k*)=h{k*)+f2{k*) 

< hike) + /2(fcc/2). 
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Above, the first inequality is by the optimality of kc and the second by the monotonicity of /i and / 2 . Thus, 
if 1 < fee < or equivalently if 


m > max |c' log (Cig/^) , c' log (J/Cig) | 


this yields 


1/2 


X - x \\2 < Ci5(5'-+‘”1 

-t + l/2 


d log N 


(t-l/2)(r-l/2) 
t + r-1 


^25 = oec^is ■ By Theoremini the above bound holds with probability exceeding 

1 — g-pc^'ciog-^ = 1 _ exp (^—pc m’-+‘-i logi'+i-i N^. Setting Cu = max|(C'i8/^)*“^/^ , (S/Cis) |, 
a = we obtain (l34l) . 

□ 


Remark 12.1. The results above place no restrictions on how large m can grow, compared to N. In particular, 
our results apply to both the “undersampling” {m < N) and “oversampling” m > N cases. However, they 
are not necessarily optimal for the m > N case. In this setting, the Sobolev dual decoder proposed in [5], 
when applied to SA quantized sub-Gaussian measurements of compressible signals yields an error decay of 
order where a G (0,1) controls the probability [27l|37]. This error is better than predicted by 

Corollary [TH Below, we show that our one-stage decoder also yields this quantization error decay in the 
oversampled case, while maintaining robustness to noise. 

Corollary 13. Assume all the variables are as defined in Theorem\^ with m> i> dN, where d>lS is a 
constant. Then there exists a constant c such that decoding via (US yields the uniform error bound 

II- II ^ [m 

\\x - x \\2 < Ce iy—j 6 + Cs^je, 

with probability 1 — 

Proof. One can repeat the proof of Theorem |9l with minor modifications, to obtain the result. Specifically, 
applying Proposition |S] with t = we see that with probability exceeding 1 — , the matrix <I> 

(as defined in the proof of Theorem 0) satisfies the restricted isometry property with constant S = ^ < 
2 — and sparsity k = N. Everything else remains unchanged and yields the desired result by setting 

CTk{x) = 0 . □ 


4 Numerical Experiments 

In this section, we present numerical experiments to illustrate our theoretical results. In all the simulations 
below, we use random Gaussian matrices as sensing matrices. 

Sparse signals 

In the first experiment, we compare the empirical decay rate of the reconstruction error due to solving (jl3|) . 
with those predicted in Theorem^ We generate a Gaussian matrix $0 of size 1000 x 512. We then generate 
m X 512 mesurement matrices for various values of m between 100 and 1000 by selecting the first m rows of 
<I>o- We also generate a set of 100 /c-sparse vectors, k = 10, where the support is chosen uniformly at random 
and the magnitudes are such that the restriction of x to its support is uniformly distributed in the unit-ball 
of We set the quantization step size 5 = 0.01 and use rth-order EA quantization, r = 1,2, with the 
corresponding greedy quantization rule to quantize the m compressed sensing measurements of each of the 
100 sparse vectors. Figure [T] shows the worst-case reconstruction error over the 100 experiments described 
above as a function of the number of measurements m. Note that the behavior of the error is as expected. 
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Figure 1: Worst case reconstruction error of (IT^ on a log-log plot, with parameters: N = 512; m = 
for I = 0,1,.., 10; fc = 10; (5 = 0.01. 

One-bit quantization 

Next, we repeat this experiment (albeit with N = 256, fc = 5) for one-bit SA quantization of compressed 
sensing measurements, using (fT^ with the one-bit alphabet {±1} as the quantizer and (fT4)) as the decoder. 
To ensure stability of the quantizer for r = 1,2, we consider sparse signals lying inside the £ 2 -ball with 
radius 0.15 (alternatively, we could scale the alphabet and work with vectors in the unit ball). Figure [2] 
plots the corresponding result with each point representing the worst case error over 100 experiments. The 
predicted rates, and appear in the Figure[2]as m gets large. The initial plateaus of both curves 

correspond to the range of m for which the zero-vector is feasible and thus optimal. As m increases to a 
point where zero is no longer feasible to m the predicted decay rate appears. 



Figure 2: Log-log plot for the worst case reconstruction error of one-bit quantization with parameters 
N = 256, m = [lO^+O-i'J ,1 = 7,..., 15, and fc = 5. 
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Compressible signals 

In the next experiment, we consider compressible signals. Our measurement setup is identical to that of the 
first experiment. Specifically, we use restrictions of a 1000 x 512 Gaussian matrix <i>o to generate m x 512 
measurement matrices for various values of m between 100 and 1000. We generate 100 compressible signals in 
the unit w£p ball with p = 0.5, so \x(^i) \ < where ^(i) denotes the largest component of x in magnitude. 
Specifically, each compressible signal is generated by randomly permuting the indices of a vector whose z’th 
element is drawn from the uniform distribution on the interval [—1/z^, 1/*^]. Figure [3] compares the decay 
rate (as a function of m) of the worst case error over the 100 signals with those predicted in Corollary [121 
i.e., and for r = 1,2, respectively. 



Figure 3: Log-log plot for the worst case reconstruction error of compressible signals in the unit wl^ ball 
withp= 1/2, N = 512, m= [lO^+^'^'J, 1 = 0,1,..., 10, and k = 10. 


Robustness to noise 

This experiments illustrates our decoder’s robustness to noise. We use the same set of parameters as in 
the first experiment, except that now the measurements are damped by i.i.d., uniform noise rj with ||?7||oo < 
2 • 10“^. Figure [4] provides evidence that as m grows, the limit of the reconstruction error approaches a fixed 
value as predicted in ([M|) and (l27|l of Corollary ITOl 

Root-exponential error-decay 

Our final experiment shows the root exponential decay predicted in Corolloarv llll Here, for the compressed 
sensing measurements of each generated signal, we apply EA quantization with different orders (from one 
to five) and select the one with the smallest reconstruction error. For this experiment, the setup is again 
similar to our first experiment, with m ranging from 100 to 398 while N = 512, fc = 5, and S = 0.01. Since 
the sparsity level is fixed, the oversampling factor A is propositional to m and we expect the reconstruction 
error to decay root exponentially in m, 

\\xopt-x\\<e-^^, 

or equivalently 

log(-log(IIXopt - a;||)) > - log m-i-log c. 

Thus, in Figure |5l we plot (as a function of m), the log(—log(-)) of the mean reconstruction error over 50 
experiments as well as ^ log m + log c. 
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Figure 4: Worst case reconstruction error with noisy measurements setting e = 2 • 10 iV = 512, m = 
[lO^+o i'J, / = 0,1,10, and k = 10. 



Figure 5: Root exponential decay, with N = 512; k = 5; N = 512, and m = [10^+°-^^J, / = 0,1,..., 10 . 


5 Acknowledgements 

R. Wang was funded in part by an NSERC Collaborative Research and Development Grant DNOISE II 
(22R07504). O. Yihnaz was funded in part by a Natural Sciences and Engineering Research Council of 
Canada (NSERC) Discovery Grant (22R82411), an NSERC Accelerator Award (22R68054) and an NSERC 
Collaborative Research and Development Grant DNOISE II (22R07504). 


References 

[1] Waheed U Bajwa, Jarvis D Haupt, Gil M Raz, Stephen J Wright, and Robert D Nowak. Toeplitz- 
structured compressed sensing matrices. In Statistical Signal Processing, 2007. SSP’07. lEEE/SP 14th 
Workshop on, pages 294-298. IEEE, 2007. 


18 















[2] Richard Baraniuk, Simon Foucart, Deanna Needell, Yaniv Plan, and Mary Wootters. Exponential decay 
of reconstruction error from binary measurements of sparse signals. arXiv preprint arXiv:1407.8246^ 
2014. 

[3] Richard G Baraniuk, Volkan Cevher, Marco F Duarte, and Chinmay Hegde. Model-based compressive 
sensing. IEEE Trans. Inf. Theory, 56(4):1982~2001, 2010. 

[4] J.J. Benedetto, A.M. Powell, and O. Yilmaz. Sigma-delta (EA) quantization and finite frames. IEEE 
Trans. Inf. Theory, 52(5):1990-2005, 2006. 

[5] J. Blum, M. Lammers, A.M. Powell, and O. Yilmaz. Sobolev duals in frame theory and sigma-delta 
quantization. J. Fourier Anal. AppL, 16(3):365-381, 2010. 

[6] T. Blumensath and M.E. Davies. Iterative hard thresholding for compressed sensing. Applied and 
Computational Harmonic Analysis, 27(3):265-274, 2009. 

[7] B.G. Bodmann, V.l. Paulsen, and S.A. Abdulbaki. Smooth frame-path termination for higher order 
sigma-delta quantization. J. Fourier Anal. AppL, 13(3):285“307, 2007. 

[8] P. T. Boufounos. Universal rate-efficient scalar quantization. IEEE Trans. Inf. Theory, 58(3):1861-1872, 
2012 . 

[9] Petros T Boufounos, Laurent Jacques, Felix Krahmer, and Rayan Saab. Quantization and compressive 
sensing. arXiv preprint arXiv:1405.1194, 2014. 

[10] P.T. Boufounos and R.G. Baraniuk. 1-bit compressive sensing. In Information Sciences and Systems, 
2008. CISS 2008. 42nd Annual Conference on, pages 16-21. IEEE, 2008. 

[11] E. J. Gandes, J. Romberg, and T. Tao. Stable signal recovery from incomplete and inaccurate measure¬ 
ments. Comm. Pure AppL Math., 59:1207-1223, 2006. 

[12] Emmanuel Gandes and Justin Romberg. Encoding the ip ball from limited measurements. In Proceedings 
of Data Compression Conference (DCC), pages 33-42, 2006. 

[13] Peter G Casazza and Gitta Kutyniok. Finite frames. Springer, 2012. 

[14] E. Chou and Giintrirk. Distributed noise-shaping quantization: 1. beta duals of finite frames and near- 
optimal quantization of random measurements. arXiv preprint arXiv:1405.4628, 2015. 

[15] E. Chou, C. S. Giintiirk, F. Krahmer, R. Saab, and Q. Yilmaz. Noise-shaping quantization methods for 
frame-based and compressive sampling systems. arXiv preprint arXiv:1502.05807, 2015. 

[16] W. Dai and O. Milenkovic. Subspace pursuit for compressive sensing signal reconstruction. Information 
Theory, IEEE Transactions on, 55(5):2230-2249, 2009. 

[17] 1. Daubechies and R. DeVore. Approximating a bandlimited function using very coarsely quantized 
data: a family of stable sigma-delta modulators of arbitrary order. Ann. Math., 158(2):679-710, 2003. 

[18] Ingrid Daubechies, Ronald A DeVore, CS Gunturk, and Vinay A Vaishampayan. A/d conversion with 
imperfect quantizers. IEEE Trans. Inf. Theory, 52(3):874-885, 2006. 

[19] P. Deift, C. S. Gunturk, and F. Krahmer. An optimal family of exponentially accurate one-bit sigma- 
delta quantization schemes. Comm. Pure AppL Math., 64(7):883-919, 2011. 

[20] D.L. Donoho. Compressed sensing. IEEE Trans. Inf. Theory, 52(4):1289-1306, 2006. 

[21] D.L. Donoho, A. Maleki, and A. Montanari. Message-passing algorithms for compressed sensing. Pro¬ 
ceedings of the National Academy of Sciences, 106(45):18914-18919, 2009. 


19 


[22] J. Feng and F. Krahmer. An RIP-based approach to EA quantization for compressed sensing. IEEE 
Signal Process. Lett., 21(11):1351-1355, 2014. 

[23] S. Foucart. Stability and robustness of £i-minimizations with Weibull matrices and redundant dictio¬ 
naries. Linear Algebra and its Applications, 441:4-21, 2014. 

[24] M.P. Friedlander, H. Mansour, R. Saab, and O. Yilmaz. Recovering compressively sampled signals using 
partial support information. IEEE Trans. Inf. Theory, 58(2):1122-1134, Feb 2012. 

[25] V.K. Goyal, M. Vetterli, and N.T. Thao. Quantized overcomplete expansions in M.^: analysis, synthesis, 
and algorithms. IEEE Trans. Inf. Theory, 44(1):16-31, Jan 1998. 

[26] C.S. Giintiirk. One-bit sigma-delta quantization with exponential accuracy. Comm. Pure Appl. Math., 
56(11):1608-1630, 2003. 

[27] C.S. Giintiirk, M. hammers, A.M. Powell, R. Saab, and O. Ydmaz. Sobolev duals for random frames and 
sigma-delta quantization of compressed sensing measurements. Foundations of Computational mathe¬ 
matics, 13(l):l-36, 2013. 

[28] H. Inose and Y. Yasuda. A unity bit coding method by negative feedback. Proceedings of the IEEE, 
51(11):1524-1535, 1963. 

[29] M. Iwen and R. Saab. Near-optimal encoding for sigma-delta quantization of finite frame expansions. 
J. Fourier Anal. Appl., pages 1-19, 2013. 

[30] L. Jacques, D. K. Hammond, and M. J. Fadili. Dequantizing Compressed Sensing: When Oversampling 
and Non-Gaussian Constraints Combine. IEEE Trans. Inf. Theory, 57(1):559-571, January 2011. 

[31] L. Jacques, D. K. Hammond, and M. J. Fadili. Stabilizing Nonuniformly Quantized Compressed Sensing 
With Scalar Companders. IEEE Trans. Inf. Theory, 59(12):7969 - 7984, January 2013. 

[32] L. Jacques, J. N. Laska, P. T. Boufounos, and R. G. Baraniuk. Robust 1-bit compressive sensing via 
binary stable embeddings of sparse vectors. IEEE Trans. Inf. Theory, 59(4):2082-2102, 2013. 

[33] Ulugbek S Kamilov, Vivek K Goyal, and Sundeep Rangan. Message-passing de-quantization with 
applications to compressed sensing. IEEE Trans. Sig. Process., 60(12):6270-6281, 2012. 

[34] Karin Knudson, Rayan Saab, and Rachel Ward. One-bit compressive sensing with norm estimation. 
arXiv preprint arXiv: 1404.6853, 2014. 

[35] Jelena Kovacevic and Amina Chebira. Life beyond bases: The advent of frames (part i). Signal 
Processing Magazine, IEEE, 24(4):86-104, 2007. 

[36] F. Krahmer, R. Saab, and R. Ward. Root-exponential accuracy for coarse quantization of finite frame 
expansions. IEEE Trans. Inf. Theory, 58(2):1069 -1079, February 2012. 

[37] F. Krahmer, R. Saab, and Q Yilmaz. Sigma-Delta quantization of sub-Gaussian frame expansions and 
its application to compressed sensing. Information and Inference, 3(l):40-58, 2013. 

[38] Felix Krahmer, Shahar Mendelson, and Holger Rauhut. Suprema of chaos processes and the restricted 
isometry property. Communications on Pure and Applied Mathematics, 2014. 

[39] M. hammers, A.M. Powell, and O. Yilmaz. Alternative dual frames for digital-to-analog conversion in 
sigma-delta quantization. Adv. Comput. Math., pages 1-30, 2008. 

[40] G.G. Lorentz, M. von Golitschek, and Y. Makovoz. Constructive approximation: advanced problems. 
Grundlehren der mathematischen Wissenschaften. Springer, 1996. 


20 


[41] H. Mansour and R. Saab. Recovery analysis for weighted ^i-minimization using a null space property. 
Arxiv preprint arXiv:1412.1565, 2012. 

[42] D. Needell and J.A. Tropp. Cosamp: Iterative signal recovery from incomplete and inaccurate samples. 
Applied and Computational Harmonic Analysis, 26(3):301-321, 2009. 

[43] S. R. Norsworthy, R. Schreier, and G. C. Temes, editors. Delta-Sigma-Converters: Theory, Design and 
Simulation. Wiley-IEEE, 1996. 

[44] Gotz E Pfander, Holger Rauhut, and Joel A Tropp. The restricted isometry property for time-frequency 
structured random matrices. Probability Theory and Related Fields, 156(3-4):707-737, 2013. 

[45] Y. Plan and R. Vershynin. One-bit compressed sensing by linear programming. Comm. Pure Appl. 
Math., 66(8):1275-1297, 2013. 

[46] A. M. Powell, R. Saab, and O Yilmaz. Quantization and finite frames. In P. G. Gasazza and G. Kutyniok, 
editors. Finite Frames, ANHA, pages 267-302. Birkhauser Boston, 2013. 

[47] Holger Rauhut, Justin Romberg, and Joel A Tropp. Restricted isometries for partial random circulant 
matrices. Applied and Computational Harmonic Analysis, 32(2):242-254, 2012. 

[48] M Rudelson and R. Vershynin. Smallest singular value of a random rectangular matrix. Communications 
on Pure and Applied Mathematics, 62(12):1707-1739, 2009. 

[49] Rayan Saab and zgr Ylmaz. Sparse recovery by non-convex optimization instance optimality. Applied 
and Computational Harmonic Analysis, 29(1):30 - 48, 2010. 

[50] Carsten Schiitt. Entropy numbers of diagonal operators between symmetric banach spaces. Journal of 
approximation theory, 40(2):121-128, 1984. 

[51] R. Vershynin. Introduction to the non-asymptotic analysis of random matrices. In Y.G. Eldar and 
G. Kutyniok, editors. Compressed Sensing: Theory and Applications, pages xii-|-544. Cambridge Univ 
Press, Cambridge, 2012. 

[52] Ozgiir Yilmaz. Stability analysis for several second-order sigma?delta methods of coarse quantization 
of bandlimited functions. Constructive approximation, 18(4):599-623, 2002. 


21 


